Stochastic Zeroth-Order Riemannian Derivative Estimation and Optimization

We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve problems with only noisy objective function evaluations. Toward this, our main contribution propose estimators of gradient and Hessian from evaluations, based on a version Gaussian smoothing technique. The proposed overcome difficulty nonlinearity manifold constraint issues that arise using techniques when defined manifold. use following settings for function: (i) gradient-Lipschitz (in both nonconvex geodesic convex settings), (ii) sum nonsmooth functions, (iii) Hessian-Lipschitz. For these settings, we analyze oracle complexity algorithms obtain appropriately notions ?-stationary point or ?-approximate local minimizer. Notably, complexities are independent dimension ambient space depend intrinsic under consideration. demonstrate applicability by simulation results real-world applications black-box stiffness control robotics attacks neural networks. Funding: J. Li S. Ma acknowledge support National Science Foundation (NSF) [Grants DMS-1953210, CCF-1934568, CCF-2007797]. K. Balasubramanian acknowledges NSF [Grant DMS-2053918]. UC Davis CeDAR (Center Data Artificial Intelligence Research) Innovative Seed Funding Program.